Multiple prediction blocks for one intra-coded block

ABSTRACT

Devices, systems and methods for digital video processing, which includes multiple prediction blocks for one intra-coded block, are described. In a representative aspect, a method for video processing includes generating a final prediction block for a conversion between a current block of visual media data and a bitstream representation of the current block and performing the conversion based on the final prediction block. At least a portion of the final prediction block is generated based on a combination of a first prediction block and a second prediction block that are based on reconstructed samples from an image segment that comprises the current block.

CROSS REFERENCE TO RELATED APPLICATIONS

This application is a continuation of International Application No.PCT/IB2019/057904, filed on Sep. 19, 2019, which claims the priority toand benefit of International Patent Application No. PCT/CN2018/106518,filed on Sep. 19, 2018. All the aforementioned patent applications arehereby incorporated by reference in their entireties.

TECHNICAL FIELD

This patent document relates to video coding techniques, devices andsystems.

BACKGROUND

In spite of the advances in video compression, digital video stillaccounts for the largest bandwidth use on the internet and other digitalcommunication networks. As the number of connected user devices capableof receiving and displaying video increases, it is expected that thebandwidth demand for digital video usage will continue to grow.

SUMMARY

Devices, systems and methods related to digital video coding, andspecifically, intra mode coding for images and video based of past(e.g., historical or statistical) information are described. Thedescribed methods may be applied to both the existing video codingstandards (e.g., High Efficiency Video Coding (HEVC)) and future videocoding standards (e.g., Versatile Video Coding (VVC)) or codecs.

In one representative aspect, the disclosed technology may be used toprovide a method for video processing. This example method includesselecting, for a conversion between a current block of visual media dataand a bitstream representation of the current block, a first intraprediction mode based on at least a first set of history intra codinginformation that includes statistical information of a set of intraprediction modes, and performing the conversion based on the first intraprediction mode.

In another representative aspect, the disclosed technology may be usedto provide a method for video processing. This example method includes,selecting, for a conversion between a current block of visual media dataand a bitstream representation of the current block, a first intraprediction mode based on at least intra prediction modes associated withnon-adjacent neighbors of the current block. The method also includesperforming the conversion based on the first intra prediction mode.

In another representative aspect, the disclosed technology may be usedto provide a method for video processing. This example method includesselecting, for a conversion between a current block of visual media dataand a bitstream representation of the current block, an intra predictionmode based on at least one of spatial neighboring blocks to the currentblock, and performing the conversion based on the intra prediction mode.The at least one of the spatial neighboring blocks is different from afirst block that is located to a left of a first row of the currentblock and a second block that is located above a first column of thecurrent block.

In another representative aspect, the disclosed technology may be usedto provide a method for video processing. This example method includesgenerating a final prediction block for a conversion between a currentblock of visual media data and a bitstream representation of the currentblock and performing the conversion based on the final prediction block.At least a portion of the final prediction block is generated based on acombination of a first prediction block and a second prediction blockthat are based on reconstructed samples from an image segment thatcomprises the current block.

In another representative aspect, the disclosed technology may be usedto provide a method for video processing. This example method includesmaintaining a first table of motion candidates during a conversionbetween a current block of visual media data and a bitstreamrepresentation of the current block and determining, based on at leastthe first table of motion candidates, motion information for the currentblock that is coded using an intra block copy (IBC) mode in which atleast one motion vector is directed at an image segment that comprisesthe current block. The method also includes performing the conversionbased on the motion information.

In another representative aspect, the disclosed technology may be usedto provide a method for video coding. This example method for videocoding includes selecting, for a bitstream representation of a currentblock of visual media data, a first intra prediction mode from a set ofintra prediction modes based on a first set of past intra codinginformation, and processing, based on the first intra prediction mode,the bitstream representation to generate the current block.

In another representative aspect, the disclosed technology may be usedto provide a method for video coding. This example method for videocoding includes selecting, for a bitstream representation of a currentblock of visual media data, an intra prediction mode from at least oneof adjacent or non-adjacent spatial neighboring blocks to the currentblock, and processing, based on the intra prediction mode, the bitstreamrepresentation to generate the current block.

In yet another representative aspect, the disclosed technology may beused to provide a method for video coding. This example method for videocoding includes generating, for a bitstream representation of a currentblock of visual media data, a first prediction block and a secondprediction block that are based on reconstructed samples from an imagesegment that comprises the current block, generating at least a portionof a final prediction block based on a linear function of the first andsecond prediction blocks, and processing, based on the final predictionblock, the bitstream representation to generate the current block.

In yet another representative aspect, the above-described method isembodied in the form of processor-executable code and stored in acomputer-readable program medium.

In yet another representative aspect, a device that is configured oroperable to perform the above-described method is disclosed. The devicemay include a processor that is programmed to implement this method.

In yet another representative aspect, a video decoder apparatus mayimplement a method as described herein.

The above and other aspects and features of the disclosed technology aredescribed in greater detail in the drawings, the description and theclaims.

BRIEF DESCRIPTION OF THE DRAWINGS

FIG. 1 shows an example of 67 intra prediction modes.

FIG. 2 shows an example of neighboring blocks used for intra modeprediction.

FIGS. 3A and 3B show examples of reference samples for wide-angle intraprediction modes for non-square blocks.

FIG. 4 shows an example of a discontinuity when using wide-angle intraprediction.

FIGS. 5A-5D show examples of samples used by a position-dependent intraprediction combination (PDPC) method.

FIG. 6 shows an example of locations of samples used for the derivationof the weights of the linear model.

FIG. 7 shows an example of classifying neighboring samples into twogroups.

FIG. 8A shows an example of a chroma sample and its corresponding lumasamples.

FIG. 8B shows an example of down filtering for the cross-componentlinear model (CCLM) in the Joint Exploration Model (JEM).

FIGS. 9A and 9B show examples of a chroma block and its correspondingluma block.

FIG. 10 shows an example of intra prediction modes with default modes.

FIGS. 11A and 11B show examples of corresponding sub-blocks for a chromaCB.

FIG. 12 shows an example of an intra-picture block copy.

FIGS. 13A-13F show examples of different adjacent blocks that may beselected for intra mode coding.

FIG. 14 shows a flowchart of an example method for intra-mode codingbased on past information in accordance with the disclosed technology.

FIG. 15 shows a flowchart of another example method for intra-modecoding based on past information in accordance with the disclosedtechnology.

FIG. 16 shows a flowchart of yet another example method for intra-modecoding based on past information in accordance with the disclosedtechnology.

FIG. 17 is a block diagram of an example of a hardware platform forimplementing a visual media decoding or a visual media encodingtechnique described in the present document.

FIG. 18 is a block diagram of an example video processing system inwhich disclosed techniques may be implemented.

FIG. 19 shows a flowchart of an example method for video processing inaccordance with the disclosed technology.

FIG. 20 shows a flowchart of another example method for video processingin accordance with the disclosed technology.

FIG. 21 shows a flowchart of another example method for video processingin accordance with the disclosed technology.

FIG. 22 shows a flowchart of another example method for video processingin accordance with the disclosed technology.

FIG. 23 shows a flowchart of another example method for video processingin accordance with the disclosed technology.

DETAILED DESCRIPTION

Due to the increasing demand of higher resolution video, video codingmethods and techniques are ubiquitous in modern technology. Video codecstypically include an electronic circuit or software that compresses ordecompresses digital video, and are continually being improved toprovide higher coding efficiency. A video codec converts uncompressedvideo to a compressed format or vice versa. There are complexrelationships between the video quality, the amount of data used torepresent the video (determined by the bit rate), the complexity of theencoding and decoding algorithms, sensitivity to data losses and errors,ease of editing, random access, and end-to-end delay (latency). Thecompressed format usually conforms to a standard video compressionspecification, e.g., the High Efficiency Video Coding (HEVC) standard(also known as H.265 or MPEG-H Part 2), the Versatile Video Coding (VVC)standard to be finalized, or other current and/or future video codingstandards.

Embodiments of the disclosed technology may be applied to existing videocoding standards (e.g., HEVC, H.265) and future standards to improveruntime performance. Section headings are used in the present documentto improve readability of the description and do not in any way limitthe discussion or the embodiments (and/or implementations) to therespective sections only.

1 Examples of intra prediction in VVC1.1 Intra mode coding with 67 intra prediction modes

To capture the arbitrary edge directions presented in natural video, thenumber of directional intra modes is extended from 33, as used in HEVC,to 65. The additional directional modes are depicted as red dottedarrows in FIG. 1, and the planar and DC modes remain the same. Thesedenser directional intra prediction modes apply for all block sizes andfor both luma and chroma intra predictions.

Conventional angular intra prediction directions are defined from 45degrees to −135 degrees in clockwise direction as shown in FIG. 1. InVTM2, several conventional angular intra prediction modes are adaptivelyreplaced with wide-angle intra prediction modes for the non-squareblocks. The replaced modes are signaled using the original method andremapped to the indexes of wide angular modes after parsing. The totalnumber of intra prediction modes is unchanged (e.g., 67), and the intramode coding is unchanged.

In HEVC, every intra-coded block has a square shape and the length ofeach of its side is a power of 2. Thus, no division operations arerequired to generate an intra-predictor using DC mode. In VTV2, blockscan have a rectangular shape that necessitates the use of a divisionoperation per block in the general case. To avoid division operationsfor DC prediction, only the longer side is used to compute the averagefor non-square blocks.

1.2 Examples of Intra Mode Coding

In some embodiments, and to keep the complexity of the MPM listgeneration low, an intra mode coding method with 3 Most Probable Modes(MPMs) is used. The following three aspects are considered to the MPMlists:

-   -   Neighbor intra modes;    -   Derived intra modes; and    -   Default intra modes.

For neighbor intra modes (A and B), two neighbouring blocks, located inleft and above are considered. The left and above blocks are those whoare connected to the top-left sample of current block, as shown in FIG.2. An initial MPM list is formed by performing pruning process for twoneighboring intra modes. The pruning process is used to removeduplicated modes so that only unique modes can be included into the MPMlist. If two neighboring modes are different each other, one of thedefault modes (e.g., PLANA (0), DC (1), ANGULAR50 (e.g., 50)) is addedto the MPM list after the pruning check with the existing two MPMs. Whenthe two neighboring modes are the same, either the default modes or thederived modes are added to the MPM list after the pruning check. Thedetailed generation process of three MPM list is derived as follows:

If two neighboring candidate modes (i.e., A==B) are same,

-   -   If A is less than 2, candModeList [3]={0, 1, 50}.    -   Otherwise, candModeList[0]={A, 2+((A+61) % 64), 2+((A−1) % 64)}

Otherwise,

-   -   If neither of A and B is equal to 0, candModeList [3]={A, B, 0}.    -   Otherwise, if neither of A and B is equal to 1, candModeList        [3]={A, B, 1}.    -   Otherwise, candModeList [3]={A, B, 50}.

An additional pruning process is used to remove duplicated modes so thatonly unique modes can be included into the MPM list. For entropy codingof the 64 non-MPM modes, a 6-bit Fixed Length Code (FLC) is used.

1.3 Wide-angle intra prediction for non-square blocks

In some embodiments, conventional angular intra prediction directionsare defined from 45 degrees to −135 degrees in clockwise direction. InVTM2, several conventional angular intra prediction modes are adaptivelyreplaced with wide-angle intra prediction modes for non-square blocks.The replaced modes are signaled using the original method and remappedto the indexes of wide angular modes after parsing. The total number ofintra prediction modes for a certain block is unchanged, e.g., 67, andthe intra mode coding is unchanged.

To support these prediction directions, the top reference with length 2W+1, and the left reference with length 2H+1, are defined as shown inthe examples in FIGS. 3A and 3B.

In some embodiments, the mode number of replaced mode in wide-angulardirection mode is dependent on the aspect ratio of a block. The replacedintra prediction modes are illustrated in Table 1.

TABLE 1 Intra prediction modes replaced by wide-angle modes ConditionReplaced intra prediction modes W/H == 2 Modes 2, 3, 4, 5, 6, 7 W/H > 2Modes 2, 3, 4, 5, 6, 7, 8, 9, 10, 11 W/H == 1 None H/W == 1/2 Modes 61,62, 63, 64, 65, 66 H/W < 1/2 Mode 57, 58, 59, 60, 61, 62, 63, 64, 65, 66

As shown in FIG. 4, two vertically-adjacent predicted samples may usetwo non-adjacent reference samples in the case of wide-angle intraprediction. Hence, low-pass reference samples filter and side smoothingare applied to the wide-angle prediction to reduce the negative effectof the increased gap Δp_(α).

1.4 Examples of Position Dependent Intra Prediction Combination (PDPC)

In the VTM2, the results of intra prediction of planar mode are furthermodified by a position dependent intra prediction combination (PDPC)method. PDPC is an intra prediction method which invokes a combinationof the un-filtered boundary reference samples and HEVC style intraprediction with filtered boundary reference samples. PDPC is applied tothe following intra modes without signaling: planar, DC, horizontal,vertical, bottom-left angular mode and its eight adjacent angular modes,and top-right angular mode and its eight adjacent angular modes.

The prediction sample pred(x,y) is predicted using an intra predictionmode (DC, planar, angular) and a linear combination of reference samplesaccording to the Equation as follows:

pred(x,y)=(wL×R _(−1y) +wT×R _(x,−1) −wTL×R_(−1,−1)+(64−wL−wT+wTL)×pred(x,y)+32)>>shift

Herein, R_(x,−1), R_(−1,y) represent the reference samples located atthe top and left of current sample (x,y), respectively, and R_(−1,−1)represents the reference sample located at the top-left corner of thecurrent block.

In some embodiments, and if PDPC is applied to DC, planar, horizontal,and vertical intra modes, additional boundary filters are not needed, asrequired in the case of HEVC DC mode boundary filter orhorizontal/vertical mode edge filters.

FIGS. 5A-5D illustrate the definition of reference samples (R_(x,−1),R_(−1,y) and R_(−1,−1)) for PDPC applied over various prediction modes.The prediction sample pred (x′, y′) is located at (x′, y′) within theprediction block. The coordinate x of the reference sample R_(x,−1) isgiven by: x=x′+y′+1, and the coordinate y of the reference sampleR_(−1,y) is similarly given by: y=x′+y′+1.

In some embodiments, the PDPC weights are dependent on prediction modesand are shown in Table 2, where S=shift.

TABLE 2 Examples of PDPC weights according to prediction modesPrediction modes wT wL wTL Diagonal top-right 16 >> ((y′ << 1) >> S)16 >> ((x′ << 1) >> S) 0 Diagonal bottom-left 16 >> ((y′ << 1) >> S)16 >> ((x′ << 1) >> S) 0 Adjacent diag. top- 32 >> ((y′ << 1) >> S) 0 0right Adjacent diag. 0 32 >> ((x′ << 1) >> S) 0 bottom-left

1.5 Examples of Chroma Coding

In HEVC chroma coding, five modes (including one direct mode (DM) whichis the intra prediction mode from the top-left corresponding luma blockand four default modes) are allowed for a chroma block. The two colorcomponents share the same intra prediction mode.

In some embodiments, and different from the design in HEVC, two newmethods have been proposed, including: cross-component linear model(CCLM) prediction mode and multiple DMs.

1.5.1 Examples of the Cross-Component Linear Model (CCLM)

In some embodiments, and to reduce the cross-component redundancy, across-component linear model (CCLM) prediction mode (also referred to asLM), is used in the JEM, for which the chroma samples are predictedbased on the reconstructed luma samples of the same CU by using a linearmodel as follows:

pred_(C)(i,j)=α·rec_(L)′(i,j)+β  (1)

Here, pred_(C) (i,j) represents the predicted chroma samples in a CU andrec_(L)′(i,j) represents the downsampled reconstructed luma samples ofthe same CU for color formats 4:2:0 or 4:2:2 while rec_(L)′(i,j)represents the reconstructed luma samples of the same CU for colorformat 4:4:4. CCLM parameters α and β are derived by minimizing theregression error between the neighboring reconstructed luma and chromasamples around the current block as follows:

$\begin{matrix}{\alpha = {\frac{{N \cdot {\sum\left( {{L(n)} \cdot {C(n)}} \right)}} - {\sum{{L(n)} \cdot {\sum{C(n)}}}}}{{N \cdot {\sum\left( {{L(n)} \cdot {L(n)}} \right)}} - {\sum{{L(n)} \cdot {\sum{L(n)}}}}}\mspace{14mu} {and}}} & (2) \\{\beta = {\frac{{\sum{C(n)}} - {\alpha \cdot {\sum{L(n)}}}}{N}.}} & (3)\end{matrix}$

Here, L(n) represents the down-sampled (for color formats 4:2:0 or4:2:2) or original (for color format 4:4:4) top and left neighboringreconstructed luma samples, C(n) represents the top and left neighboringreconstructed chroma samples, and value of N is equal to twice of theminimum of width and height of the current chroma coding block.

In some embodiments, and for a coding block with a square shape, theabove two equations are applied directly. In other embodiments, and fora non-square coding block, the neighboring samples of the longerboundary are first subsampled to have the same number of samples as forthe shorter boundary. FIG. 6 shows the location of the left and abovereconstructed samples and the sample of the current block involved inthe CCLM mode.

In some embodiments, this regression error minimization computation isperformed as part of the decoding process, not just as an encoder searchoperation, so no syntax is used to convey the α and β values.

In some embodiments, the CCLM prediction mode also includes predictionbetween the two chroma components, e.g., the Cr (red-difference)component is predicted from the Cb (blue-difference) component. Insteadof using the reconstructed sample signal, the CCLM Cb-to-Cr predictionis applied in residual domain. This is implemented by adding a weightedreconstructed Cb residual to the original Cr intra prediction to formthe final Cr prediction:

pred_(Cr)*(i,j)=pred_(Cr)(i,j)+α·resi_(Cb)′(i,j)  (4)

Here, resi_(Cb)′(i,j) presents the reconstructed Cb residue sample atposition (i,j).

In some embodiments, the scaling factor α may be derived in a similarway as in the CCLM luma-to-chroma prediction. The only difference is anaddition of a regression cost relative to a default α value in the errorfunction so that the derived scaling factor is biased towards a defaultvalue of −0.5 as follows:

$\begin{matrix}{\alpha = \frac{\begin{matrix}{{N \cdot {\sum\left( {{{Cb}(n)} \cdot {{Cr}(n)}} \right)}} -} \\{{\sum{{Cb}{(n) \cdot {\sum{{Cr}(n)}}}}} + {\lambda \cdot \left( {- 0.5} \right)}}\end{matrix}}{{N \cdot {\sum\left( {{{Cb}(n)} \cdot {{Cb}(n)}} \right)}} - {\sum{{{Cb}(n)} \cdot {\sum{{Cb}(n)}}}} + \lambda}} & (5)\end{matrix}$

Here, Cb(n) represents the neighboring reconstructed Cb samples, Cr(n)represents the neighboring reconstructed Cr samples, and λ is equal toΣ(Cb(n)·Cb(n))>>9.

In some embodiments, the CCLM luma-to-chroma prediction mode is added asone additional chroma intra prediction mode. At the encoder side, onemore RD cost check for the chroma components is added for selecting thechroma intra prediction mode. When intra prediction modes other than theCCLM luma-to-chroma prediction mode is used for the chroma components ofa CU, CCLM Cb-to-Cr prediction is used for Cr component prediction.

1.5.2 Examples of a Multiple Model CCLM

In the JEM, there are two CCLM modes: the single model CCLM mode and themultiple model CCLM mode (MMLM). As indicated by the name, the singlemodel CCLM mode employs one linear model for predicting the chromasamples from the luma samples for the whole CU, while in MMLM, there canbe two models.

In MMLM, neighboring luma samples and neighboring chroma samples of thecurrent block are classified into two groups, each group is used as atraining set to derive a linear model (i.e., a particular α and β arederived for a particular group). Furthermore, the samples of the currentluma block are also classified based on the same rule for theclassification of neighboring luma samples.

FIG. 7 shows an example of classifying the neighboring samples into twogroups. Threshold is calculated as the average value of the neighboringreconstructed luma samples. A neighboring sample withRec′_(L)[x,y]<=Threshold is classified into group 1; while a neighboringsample with Rec′_(L)[x,y]>Threshold is classified into group 2.

$\quad\begin{matrix}\left\{ \begin{matrix}{{{Pred}_{c}\left\lbrack {x,y} \right\rbrack} = {{\alpha_{1} \times {{Rec}_{L}^{\prime}\left\lbrack {x,y} \right\rbrack}} + \beta_{1}}} & {{{if}\mspace{14mu} {{Rec}_{L}^{\prime}\left\lbrack {x,y} \right\rbrack}} \leq {Threshold}} \\{{{Pred}_{c}\left\lbrack {x,y} \right\rbrack} = {{\alpha_{2} \times {{Rec}_{L}^{\prime}\left\lbrack {x,y} \right\rbrack}} + \beta_{2}}} & {{{if}\mspace{14mu} {{Rec}_{L}^{\prime}\left\lbrack {x,y} \right\rbrack}} > {Threshold}}\end{matrix} \right. & (6)\end{matrix}$

FIGS. 8A and 8B show an example of a chroma sample and its correspondingluma samples, and an example of of down filtering for thecross-component linear model (CCLM) in the Joint Exploration Model(JEM), respectively.

1.5.3 Examples of Chroma Coding in VVC

With regard to existing implementations, CCLM as in JEM is adopted inVTM-2.0, whereas MM-CCLM in JEM has not been adopted in VTM-2.0.

In some embodiments, for chroma intra coding, a candidate list of chromaintra prediction modes may be derived first, and wherein three parts maybe included:

-   -   (1) One direct mode (DM) which is set to the one intra luma        prediction mode associated with luma CB covering the co-located        top-left position of a chroma block. An example is shown in        FIGS. 9A and 9B with the co-located position denoted as “TL”    -   (2) One cross-component linear model (CCLM) mode    -   (3) Four default modes (DC, Planar, Horizontal, and Vertical        modes, as highlighted in FIG. 1). In an example, if one of the        four default modes is identical to the DM mode, it is replaced        by the intra prediction mode with largest mode index, e.g.,        indicated by the dashed line depicted in FIG. 10.

1.6 Examples of Syntax, Semantics and Decoding in VVC2

In some embodiments, the syntax and semantics of intra mode relatedinformation may be specified as follows:

TABLE 3 Example of a coding unit (CU) syntax coding_unit( x0, y0,cbWidth, cbHeight, treeType ) { Descriptor  if( slice_type != I ) {   pred_mode_flag ae(v)  }  if( CuPredMode[ x0 ][ y0 ] = = MODE_INTRA ){   if( treeType = = SINGLE_TREE | | treeType = = DUAL_TREE_LUMA ) {   intra_luma_mpm_flag[ x0 ][ y0 ]    if( intra_luma_mpm_flag[ x0 ][ y0] )     intra_luma_mpm_idx[ x0 ][ y0 ] ae(v)    else    intra_luma_mpm_remainder[ x0 ][ y0 ] ae(v)   }   if( treeType = =SINGLE_TREE | | treeType = = DUAL_TREE_CHROMA )   intra_chroma_pred_mode[ x0 ][ y0 ] ae(v)  } else {   [Ed. (BB): Interprediction yet to be added, pending further specification development.] }  if( CuPredMode[ x0 ][ y0 ] != MODE_INTRA )   cu_cbf ae(v)  if(cu_cbf ) {   transform_tree( x0, y0, cbWidth, cbHeight, treeType ) }

Examples of coding unit semantics. In some embodiments, the syntaxelements intra_luma_mpm_flag[x0][y0], intra_luma_mpm_idx[x0][y0] andintra_luma_mpm_remainder[x0][y0] specify the intra prediction mode forluma samples. The array indices x0, y0 specify the location (x0, y0) ofthe top-left luma sample of the considered prediction block relative tothe top-left luma sample of the picture. Whenintra_luma_mpm_flag[x0][y0] is equal to 1, the intra prediction mode isinferred from a neighbouring intra-predicted prediction unit accordingto the specification (e.g., clause 8.2.2).

In some embodiments, intra_chroma_pred_mode[x0][y0] specifies the intraprediction mode for chroma samples. The array indices x0, y0 specify thelocation (x0, y0) of the top-left luma sample of the consideredprediction block relative to the top-left luma sample of the picture.

Examples of deriving the luma intra prediction mode. In someembodiments, an input to this process is a luma location (xPb, yPb)specifying the top-left sample of the current luma prediction blockrelative to the top-left luma sample of the current picture. In thisprocess, the luma intra prediction mode IntraPredModeY[xPb][yPb] isderived. Table 4 specifies the value for the intra prediction mode andthe associated names.

TABLE 4 Example specification of intra prediction modes and associatednames Intra prediction mode Associated name 0 INTRA_PLANAR 1 INTRA_DC2..66 INTRA_ANGULAR2..INTRA_ANGULAR66 77 INTRA_CCLM

In the context of Table 4, the intra prediciton mode INTRA_CCLM is onlyapplicable to chroma components, and IntraPredModeY[xPb][yPb] labelled 0. . . 66 represents directions of predictions as shown in FIG. 1. Insome embodiments, IntraPredModeY[xPb][yPb] is derived by the followingordered steps:

-   -   1. The neighbouring locations (xNbA, yNbA) and (xNbB, yNbB) are        set equal to (xPb−1, yPb) and (xPb, yPb−1), respectively.    -   2. For X being replaced by either A or B, the variables        candIntraPredModeX are derived as follows:        -   The availability derivation process for a block is invoked            with the location (xCurr, yCurr) set equal to (xPb, yPb) and            the neighbouring location (xNbY, yNbY) set equal to (xNbX,            yNbX) as inputs, and the output is assigned to availableX.        -   The candidate intra prediction mode candIntraPredModeX is            derived as follows:            -   If one or more of the following conditions are true,                candIntraPredModeX is set equal to INTRA_DC.                -   The variable availableX is equal to FALSE.                -   CuPredMode[xNbX][yNbX] is not equal to MODE_INTRA.                -   X is equal to B and yPb−1 is less than ((yPb>>CtbLog                    2SizeY)<<CtbLog 2SizeY).                -   Otherwise, candIntraPredModeX is set equal to                    IntraPredModeY[xNbX][yNbX].    -   3. The candModeList[x] with x=0 . . . 2 is derived as follows:        -   If candIntraPredModeB is equal to candIntraPredModeA, the            following applies:            -   If candIntraPredModeA is less than 2 (i.e., equal to                INTRA_PLANAR or INTRA_DC), candModeList[x] with x=0 . .                . 2 is derived as follows:

candModeList[0]=INTRA_PLANAR  (8-1)

candModeList[1]=INTRA_DC  (8-2)

candModeList[2]=INTRA_ANGULAR50  (8-3)

-   -   -   -   Otherwise, candModeList[x] with x=0 . . . 2 is derived                as follows:

candModeList[0]=candIntraPredModeA  (8-4)

candModeList[1]=2+((candIntraPredModeA+61)% 64)  (8-5)

candModeList[2]=2+((candIntraPredModeA−1)% 64)  (8-6)

-   -   -   -   Otherwise (candIntraPredModeB is not equal to                candIntraPredModeA), the following applies:                -   candModeList[0] and candModeList[1] are derived as                    follows:

candModeList[0]=candIntraPredModeA  (8-7)

candModeList[1]=candIntraPredModeB  (8-8)

-   -   -   -   -   If neither of candModeList[0] and candModeList[1] is                    equal to INTRA_PLANAR, candModeList[2] is set equal                    to INTRA_PLANAR,                -   Otherwise, if neither of candModeList[0] and                    candModeList[1] is equal to INTRA_DC,                    candModeList[2] is set equal to INTRA_DC,                -   Otherwise, candModeList[2] is set equal to                    INTRA_ANGULAR50.

    -   4. IntraPredModeY[xPb][yPb] is derived by applying the following        procedure:        -   If intra_luma_mpm_flag[xPb][yPb] is equal to 1, the            IntraPredModeY[xPb][yPb] is set equal to            candModeList[intra_luma_mpm_idx[xPb][yPb] ].        -   Otherwise, IntraPredModeY[xPb][yPb] is derived by applying            the following ordered steps:            -   1. The array candModeList[x], x=0 . . . 2 is modified by                the following ordered steps:                -   i. When candModeList[0] is greater than                    candModeList[1], both values are swapped as follows:

(candModeList[0],candModeList[1])=Swap(candModeList[0],candModeList[1])  (8-9)

-   -   -   -   -   ii. When candModeList[0] is greater than                    candModeList[2], both values are swapped as follows:

(candModeList[0],candModeList[2])=Swap(candModeList[0],candModeList[2])  (8-10)

-   -   -   -   -   iii. When candModeList[1] is greater than                    candModeList[2], both values are swapped as follows:

(candModeList[1],candModeList[2])=Swap(candModeList[1],candModeList[2])  (8-11)

-   -   -   -   2. IntraPredModeY[xPb][yPb] is derived by the following                ordered steps:                -   i. IntraPredModeY[xPb][yPb] is set equal to                    intra_luma_mpm_remainder[xPb][yPb].                -   ii. For i equal to 0 to 2, inclusive, when                    IntraPredModeY[xPb][yPb] is greater than or equal to                    candModeList[i], the value of                    IntraPredModeY[xPb][yPb] is incremented by one.

The variable IntraPredModeY[x][y] with x=xPb . . . xPb+cbWidth−1 andy=yPb . . . yPb+cbHeight−1 is set to be equal toIntraPredModeY[xPb][yPb].

Examples of deriving the chroma intra prediction mode. In someembodiments, an input to this process is a luma location (xPb, yPb)specifying the top-left sample of the current chroma prediction blockrelative to the top-left luma sample of the current picture. In thisprocess, the chroma intra prediction mode IntraPredModeC[xPb][yPb] isderived.

In some embodiments, the chroma intra prediction modeIntraPredModeC[xPb][yPb] is derived usingintra_chroma_pred_mode[xPb][yPb] and IntraPredModeY[xPb][yPb] asspecified in Table 5 and Table 6.

TABLE 5 Example specification of IntraPredModeC[ xPb ][ yPb ] forsps_cclm_enabled_flag = 0 IntraPredModeY[ xPb ][ yPb ] Xintra_chroma_pred_mode[ xPb ][ yPb ] 0 50 18 1 ( 0 <= X <= 66 ) 0 66 0 00 0 1 50 66 50 50 50 2 18 18 66 18 18 3 1 1 1 66 1 4 0 50 18 1 X

TABLE 6 Example specification of IntraPredModeC[ xPb ][ yPb ] forsps_cclm_enabled_flag = 1 IntraPredModeY[ xPb ][ yPb ] Xintra_chroma_pred_mode[ xPb ][ yPb ] 0 50 18 1 ( 0 <= X <= 66 ) 0 66 0 00 0 1 50 66 50 50 50 2 18 18 66 18 18 3 1 1 1 66 1 4 77 77 77 77 77 5 050 18 1 X

1.7 Examples of Multiple Direct Modes (DMs)

An existing implementation (e.g., JVET-D0111) was recently adopted byJEM. Due to the separate block partitioning structure for luma andchroma components in I slices, one chroma CB may correspond to severalluma CBs. Therefore, to further improve the coding efficiency, aMultiple Direct Modes (MDM) method for chroma intra coding is proposed.In MDM, the list of chroma mode candidates includes the following threeparts:

-   -   (1) the CCLM mode    -   (2) Intra prediction modes derived from luma CBs covering the        co-located five positions of the current chroma block: the five        positions to be checked in order are defined as: center,        top-left, top-right, bottom-left and bottom-right 4×4 block        within the corresponding luma block of current chroma block for        I slices. For P and B slices, there is no need to check these        five sub-blocks since they shall be within one CB. An example of        five co-located luma positions is shown in FIGS. 11A and 11B,        they are denoted by CR, TL, TR, BL and BR, respectively.    -   (3) Chroma prediction modes from spatial neighboring blocks to        unify the luma and chroma mode derivation process (with removals        and additions notated):        -   5 chroma prediction modes from left, above, below-left,            above-right, and above-left spatial neighboring blocks of            current block, and the locations of the five spatial blocks            as in the merge mode (with Planar and DC modes being            removed)        -   With the planar and DC modes being added        -   Derived modes are added, those intra modes are obtained by            adding −1 or +1 to the angular modes which are already            included into the list        -   Default modes are added in the order of: Vertical (Mode 18),            Horizontal (Mode 50), Mode 2, Mode 34, Mode 66, Mode 10,            Mode 26 (with Mode 66, Mode 10 and Mode 26 being added)        -   The addition of the rule that if any of the four default            modes (Planar, Horizontal, Vertical and DC modes) is not            included in the list, the missing default modes are used to            replace the last one or more candidates.

In some embodiments, the pruning process is applied whenever a newchroma intra mode is added to the candidate list. In this contribution,the chroma intra mode candidate list size is always set to 10.

1.8 Examples of Intra Block Copy

In some embodiments, HEVC screen content coding (SCC) extensions employa new coding tool, Intra block copy (IBC), also named intra pictureblock compensation or current picture referencing (CPR), is a veryefficient technique in terms of coding performance improvement. IBC is ablock matching technique wherein a current prediction block is predictedfrom a reference block located in the already reconstructed regions ofthe same picture.

In IBC, a displacement vector (referred as block vector or BV, or motionvector) is used to signal the relative displacement from the position ofthe current block to that of the reference block. Further, thepreviously reconstructed reference block within the same picture isadded to the prediction errors to form the reconstructed current block.In this technique, the reference samples correspond to the reconstructedsamples of the current picture prior to in-loop filter operations. InHEVC, the in-loop filters refer to both deblocking and sample adaptiveoffset (SAO) filters. An example of the intra block compensation isshown in FIG. 12.

In some embodiments, the use of the IBC mode is signaled at bothsequence and picture level. When the IBC mode is enabled at sequenceparameter set (SPS), it can be enabled at picture level. When the IBCmode is enabled at picture level, the current reconstructed picture istreated as a reference picture. Therefore, no syntax change on blocklevel is needed in HEVC SCC on top of the existing HEVC inter mode tosignal the use of the IBC mode.

In some embodiments, the following is a list of features for the IBCmode in HEVC SCC:

-   -   All prediction unit partition modes in the HEVC inter mode are        supported in the IBC mode, e.g., 2N×2N or 2N×N or N×2N or N×N        and Asymmetrical Motion Partitioning (AMP) modes.    -   Merge and skip modes are also available for the IBC mode. The        merge candidate list construction is unified, containing merge        candidates from the neighboring positions that are either coded        in the IBC mode or the HEVC inter mode. Depending on the        selected merge index, the current block under merge or skip mode        can merge into either an IBC mode coded neighbor or otherwise an        HEVC inter mode coded one.    -   Block vector prediction and coding schemes for the IBC mode        reuse the schemes used for motion vector prediction and coding        in the HEVC inter mode (AMVP and MVD coding).    -   Residue transform, quantization and coefficient coding are the        same as in the HEVC inter mode.

In addition to the features enumerated above, some embodiments compriseadditional features (or treatments) for IBC modes, including:

-   -   The motion vector for the IBC mode, also referred as block        vector, is coded with integer-pel precision, but stored in        memory in quarter-pel precision after decoding as quarter-pel        precision is required in interpolation and deblocking stages.        When used in motion vector prediction for the IBC mode, the        stored vector predictor will be right shifted by 2.    -   A slice with the IBC mode enabled will be treated as a P or B        slice, even though no other picture than the current picture        itself is used as a reference picture.    -   Weighted prediction is disabled for the IBC mode as there was no        clear compression benefit observed.

2 Examples of Drawbacks in Existing Implementations

The current intra mode coding implementations in either JEM or VTMexhibit at least the following issues:

-   -   For luma coding, only prediction modes of adjacent spatial        neighboring blocks are considered. The lack of statistic intra        prediction information of previously coded blocks results in        limited coding performance improvements.    -   In an existing implementation, it is proposed to check more        neighboring blocks to be added to MPM list for better coding of        intra prediction modes. The complexity is also increased due to        multiple times of access of adjacent spatial neighboring blocks.    -   For chroma coding, the multiple DMs solution could bring        additional coding gains at the cost of higher complexity since        the neighboring chroma modes need to be stored which was not        needed in HEVC.    -   IBC could take care of repeated patterns which are far from the        current block and intra coding mode could take care of reducing        less spatial correlation. However, either of the two could only        be beneficial for a certain case.

3 Example Methods for Intra Mode Coding Based on Past Information

Embodiments of the presently disclosed technology overcome drawbacks ofexisting implementations, thereby providing video coding with highercoding efficiencies but lower computational complexity. Intra modecoding based on past information, and as described in the presentdocument, may enhance both existing and future video coding standards,is elucidated in the following examples described for variousimplementations. The examples of the disclosed technology provided belowexplain general concepts, and are not meant to be interpreted aslimiting. In an example, unless explicitly indicated to the contrary,the various features described in these examples may be combined.

Examples of Intra Mode Coding Based on Historical Information

Example 1. In one example, a table with statistical information of intraprediction modes (IPMs) may be maintained in the encoding/decodingprocess to help intra mode coding.

-   -   (a) In one example, the table includes the occurrence and/or        frequency of each IPM. Alternatively, the table may have entries        for a partial set of allowed IPMs.    -   (b) Partial of IPMs listed in the table may be added as new MPMs        to the MPM list.        -   (i) In one example, IPMs from spatial neighboring blocks are            firstly added to the MPM list in order. Afterwards, a few            IPMs from the table may be added in order after pruning. If            the list is not full, derived IPMs may be further added.        -   (ii) In one example, MPM list size may be extended. In this            case, IPMs from spatial neighboring blocks and derived modes            may be firstly added to the MPM list in order. Afterwards, a            few IPMs from the table may be added in order after pruning.    -   (c) IPMs included in a MPM list may be reordered based on the        table.        -   (i) In one example, the MPM list is firstly constructed            based on neighboring blocks and derived modes (e.g., as            described in Section 1.2). Afterwards, MPM list is            re-ordered based on the descending order of occurrence or            frequency associated with IPMs recorded in the table. In            such a way, a IPM in the MPM list with highest occurrence or            frequency will be adjusted to be in the first position of            the MPM list.

Example 2. Different tables may be used for different color components.

-   -   (a) For example, a table is used for luma intra-mode coding and        another table is used for chroma intra-mode coding.    -   (b) Alternatively, only one table is used for luma intra-mode        coding.    -   (c) Alternatively, whether to use multiple tables or one table        depend on whether separate coding trees are used for luma and        chroma, respectively.

Example 3. The frequency or occurrence of one mode M in the table isaccumulated by K (e.g. K=1) after decoding a block with mode M.

-   -   (a) In one example, the occurrence of a mode has a maximum value        (e.g. 255).    -   (b) If at least one occurrence of a mode reaches the maximum        value, a regulation (or normalization) process is applied.        -   (i) In one example, all occurrence number in the table is            right shift by a number such as 2.        -   (ii) In another example, all occurrence number in the table            subtract a number, such as the minimum occurrence number in            the table.

Example 4. Instead of utilizing IPMs of adjacent neighboring blocks forcoding a block, intra prediction modes of non-adjacent neighboringblocks may be utilized.

-   -   (a) IPMs of non-adjacent neighboring blocks may be added to the        MPM list. Alternatively, pruning of IPMs may be applied. If an        IPM of a non-adjacent neighboring block is different from any of        IPMs in the MPMS list, it may be added to the MPM list.    -   (b) In one example, MPM list size may be extended. In this case,        IPMs from adjacent neighboring blocks and derived modes may be        firstly added to the MPM list in order. Afterwards, IPMs from        non-adjacent neighboring blocks may be added in order after        pruning.    -   (c) IPMs of non-adjacent neighboring blocks may be used to        re-order the non-MPM IPMs. In one example, for those IPMs        associated with non-adjacent neighboring blocks, higher priority        may be assigned, e.g., less bits may be used to code them.

Example 5. Coding of an IPM associated with current block may depend onthe table, IPMs from adjacent spatial neighboring blocks, IPMs fromnon-adjacent spatial neighboring blocks.

Examples of Selection of Adjacent Neighboring Blocks

Example 6. Instead of using of the above and left neighboring blocksrelative to the top-left position of current block for intra mode coding(such as using the intra prediction modes of above and left neighboringblocks to construct a MPM list), different positions may be utilized.

-   -   (a) In one example, the above neighboring block relative to the        top-left position of current block is replaced by the above        neighboring block relative to the center position of the first        row of current block, as shown in FIGS. 13A-13C.    -   (b) In one example, the left neighboring block relative to the        top-left position of current block is replaced by the left        neighboring block relative to the center position of the first        column of current block, as shown in FIGS. 13D-13F.

Example 7. Instead of utilizing the fixed M relative neighboring blocksfor all kinds of blocks, selection of M neighboring blocks may depend onblock shape and/or block size.

-   -   (a) In one example, if the block is a square block, the intra        prediction modes of above and left neighboring blocks relative        to the to-left position of current block for intra mode coding        may be used for coding intra prediction mode of current block.    -   (b) In one example, if the block is a non-square block, the        intra prediction modes of above and/or left neighboring blocks        relative to the center position of the first row and/or the        first column of current block for intra mode coding may be used        for coding intra prediction mode of current block.

Examples of Multiple Prediction Blocks for One Intra-Coded Block

Example 8. Multiple prediction blocks may be generated withreconstructed samples from the same tile/slice/picture containingcurrent block.

-   -   (a) In one example, two prediction blocks may be firstly        generated and the final prediction block is generated from the        two prediction blocks. wherein one of them is generated by an        intra prediction mode and the other is generated by a motion        vector pointing to the same tile/slice/picture (e.g., intra        block copy; and the motion vector is also known as block vector)        containing current block.    -   (b) In one example, linear function is applied to the two        prediction blocks to derive a final intra prediction block.        -   (i) Alternatively, only partial of the final prediction            block is generated from the multiple (e.g., two) prediction            blocks and the remaining part is directly copied from one of            the multiple prediction blocks.        -   (ii) In one example, average of the two prediction blocks            (i.e., equal weights for the two prediction blocks) is used            as the final intra prediction block.        -   (iii) In one example, different weight for different            position relative to one block, and/or for different intra            prediction mode may be applied.        -   (iv) In one example, weights may be signaled in            SPS/PPS/VPS/picture or sequence or slice header/tile/groups            of CTUs/CTU/CU/PU/TU.        -   (v) Weights applied to the two prediction blocks may be            pre-defined.

Example 9. The above methods may be applied under certain conditions.

-   -   (a) It may be applied for certain color components, e.g., only        the luma component.    -   (b) It may be applied for certain block size/block shape etc.

Example 10. When IBC is treated as inter mode (i.e., with at least onemotion vector to derive the prediction block), History-based MotionVector Prediction (HMVP) methods, which utilize previously coded motioninformation for motion information prediction may be used. The HMVPmethods allow the encoding/decoding process to be performed based onhistorical data (e.g., the blocks that have been processed). One ormultiple tables with motion information from previously coded blocks aremaintained during the encoding/decoding process. Each table can beassociated with a counter to track the number of motion candidatesstored in the table. During the encoding/decoding of one block, theassociated motion information in tables may be used (e.g., by beingadded to the motion candidate lists) depending on the codingcharacteristics of the block (e.g., whether the block is IBC coded).After encoding/decoding the block, the tables may be updated based onthe coding characteristics of the block. That is, the updating of thetables is based on the encoding/decoding order. In some embodiments, thetables can be reset (e.g., removing the stored candidates and settingthe associated counter to 0) before processing a new slice, a newlargest coding unit (LCU) row, or a new tile.

-   -   (a) In one example, look-up-tables may be updated with motion        information from IBC-coded blocks.    -   (b) Alternatively, look-up-tables are not allowed to be updated        with motion information from IBC-coded blocks.    -   (c) In one example, for the IBC mode, motion information from        previously coded blocks are not allowed to predict the motion        information of IBC-coded blocks.    -   (d) In one example, only motion information from previously        IBC-coded blocks can be used to predict motion information of        the current IBC-coded block. One look-up table is used to        include only motion information from IBC-coded blocks and such        look-up table is only used for IBC-coded blocks.    -   (e) Alternatively, for the IBC mode, motion information from        previously coded blocks are allowed to predict the motion        information of IBC-coded blocks.        -   (i) In one example, a look-up table may include motion            information from IBC-coded blocks and other kinds of            inter-coded blocks. For example, if a motion candidate in            the look-up table is derived from an IBC coded block, it may            be used to encode/decode current IBC coded block.        -   (ii) Alternatively, multiple look-up tables may be required            and at least one of them only includes motion information            from IBC-coded blocks and one of them only includes motion            information from other kinds of inter-coded blocks. For            example, a first look-up table can include motion            information for an IBC-coded block. A second, different            look-up table can include motion information for a block            that is not IBC-coded (e.g., the block can be coded with an            inter mode). The first and second look-up tables can be            updated after the corresponding conversions are completed.            In some embodiments, a first subsequent conversion of an IBC            coded block can be performed using the updated first look-up            table, and a second subsequent conversion of an inter coded            block can be performed using the updated second look-up            table.

In some embodiments, before updating look-up tables as described inExample 10 by adding a motion candidate obtained from a coded block,pruning may be applied.

-   -   (a) In one example, the new motion candidate to be added need to        be pruned to all existing motion candidates in selected tables.        That is, the new motion candidate is compared against all        existing motion candidates in selected tables to make sure that        there are no duplicated candidates.    -   (b) Alternatively, the new motion candidates may be only pruned        to a certain number of existing motion candidates. For example,        it may be compared against the last m motion candidates (m is an        integer) in a look-up table before selectively adding it as the        last entry of the LUT.    -   (c) In some embodiments, the selected table is the one that is        used to construct the motion candidate list for the coded block        where the motion candidate is obtained from. In some        embodiments, the motion candidate can be used to update partial        or all of available look-up tables.

In some embodiments, the look-up tables as described in Example 10 maybe used when a block is coded with merge or AMVP mode. That is, thecurrent block can be coded using an IBC merge mode or an IBC AMVP mode.

In some embodiments, besides adjacent candidates, non-adjacentcandidates (e.g., merge candidates or AMVP candidates) can be combinedin the look-up tables.

-   -   (a) In one example, m candidates (non-adjacent and/or adjacent        candidates) and n motion candidates from look-up tables may be        added (e.g., when the list is not full). Both m and n are        positive integers (>0).    -   (b) In one example, m candidates (non-adjacent and/or adjacent        candidates) may be checked and/or n motion candidates from        look-up tables may be checked. The checked candidates may be        added when the list is not full.    -   (c) In one example, the adjacent or non-adjacent candidates and        motion candidates from look-up tables may be added in an        interleaved way.        -   (i) The order of checking the adjacent or non-adjacent            candidates is kept unchanged. One candidate is checked is            followed by a motion candidate from a look-up table.        -   (ii) The order of checking non-adjacent blocks is kept            unchanged. However, if one non-adjacent block is located            outside a certain range (e.g., outside the current LCU row),            it would be replaced by a motion candidate in a look-up            table.        -   (iii) In one example, the order of checking non-adjacent            blocks is kept unchanged. However, if one adjacent or            non-adjacent block is coded with intra mode or intra block            copy mode, it would be replaced by a motion candidate in a            look-up table (e.g., upon determining that the motion            candidate in the look-up table is different from the            adjacent or non-adjacent block).    -   (d) In one example, non-adjacent candidates have a higher        priority compared to motion candidates from look-up tables.        -   (i) In this case, all the non-adjacent blocks are checked            firstly. If the total number of candidates are still less            than the maximumly allowed number, then motion candidates            from look-up tables are further checked.        -   (ii) Alternatively, motion candidates from look-up tables            have a higher priority compared to non-adjacent candidates.            Motion candidates from look-up tables are first checked. If            the total number of candidates are still less than the            maximumly allowed number, the non-adjacent blocks are            checked to add non-adjacent candidates to the merge            candidate list.        -   (iii) Similarly, when both non-adjacent method and            look-up-table-based method are allowed in the Advanced            Motion Vector Prediction (AMVP) coding process, the priority            could be handled in a similar way as described in above            examples.

In some embodiments, similar to the usage of look-up tables with motioncandidates for motion vector prediction, it is proposed that one ormultiple look-up tables may be constructed, and/or updated to storeintra prediction modes from previously coded blocks and look-up tablesmay be used for coding/decoding an intra-coded block.

-   -   (a) Number of entries for each LUT is the same, e.g., equal to        total number of allowed intra predictions.    -   (b) A variable (i.e., cnt) may be further assigned for each        intra prediction mode to record how many times the intra        prediction mode has been used in previously coded blocks.    -   (c) When updating a look-up table with a selected intra        prediction mode, the associated cnt can be modified, such as        increasing by 1.

In some embodiments, intra prediction modes of non-adjacent blocks maybe used as intra prediction mode predictors for coding an intra-codedblock.

In some embodiments, look-up tables and non-adjacent based methods maybe jointly utilized. In one example, the intra prediction modes ineither look-up tables or non-adjacent blocks may be used in the MPM listconstruction process. Alternatively, the intra prediction modes ineither look-up tables or non-adjacent blocks may be used to re-ordernon-MPM intra prediction modes.

In some embodiments, a motion candidate list for an IBC-coded block canbe constructed as follows:

(a) If there exist unique block vectors (also referred to asdisplacement vectors or motion vectors as discussed above) that areassociated with two spatial neighboring blocks of the current block, addthem to the list.

(2) Check the last available motion candidate in the table (that is, thelatest candidate). If there is no duplicated candidate in the list, thelast available motion candidate is then added to the list.

(3) If the list is not full, add the remaining motion candidates in thelook-up table.

The examples described above may be incorporated in the context of themethods described below, e.g., methods 1400, 1500 and 1600, which may beimplemented at a video encoder and/or decoder.

FIG. 14 shows a flowchart of an example method for cross-componentprediction. The method 1400 includes, at step 1410, selecting, for abitstream representation of a current block of visual media data, afirst intra prediction mode from a set of intra prediction modes basedon a first set of past intra coding information. In some embodiments,the first set of past intra coding information includes historical intracoding information.

In some embodiments, and in the context of Example 1, the first set ofpast intra coding information comprises statistical intra codinginformation. In one example, the statistical intra coding informationcomprises a number of occurrences of each of the set of intra predictionmodes over a time duration, and the method 1400 further includes thesteps of constructing a most probable mode (MPM) list based onneighboring blocks of the current block, and reordering the MPM listbased on the number of occurrences.

The method 1400 includes, at step 1420, processing, based on the firstintra prediction mode, the bitstream representation to generate thecurrent block.

In some embodiments, and in the context of Example 2, the current blockcomprises a luma component and a chroma component, the first set of pastintra coding information is used for intra-mode coding the lumacomponent, and a second set of past intra coding information is used forintra-mode coding the chroma component. In an example, the lumacomponent is based on a first coding tree, and the chroma component isbased on a different second coding tree.

In some embodiments, and in the context of Example 3, the number ofoccurrences is incremented for an intra prediction mode of the set ofintra prediction modes that corresponds to the first intra predictionmode. Then, the method 1400 further includes the step of determiningthat at least one of the number of occurrences is equal to a maximumnumber of occurrences. In one example, the method further includes rightshifting each of the number of occurrences by a predefined number. Inanother example, the method further includes subtracting a minimumnumber of occurrences from each of the number of occurrences.

In some embodiments, and in the context of Example 4, the method 1400further includes the step of adding a first subset of the set of intraprediction modes to a most probable mode (MPM) list. In otherembodiments, the method 1400 may further include increasing a size of anMPM list, and adding a first subset of the set of intra prediction modesto the MPM list, wherein the first subset of intra prediction modescorrespond to blocks that are spatial neighbors of the current block. Inthese cases, the method 1400 may further include pruning the MPM list,and adding a second subset of the set of intra prediction modes to theMPM list, wherein an ordering of intra prediction modes in the secondsubset is maintained in the MPM list.

FIG. 15 shows a flowchart of another example method for cross-componentprediction. The method 1500 includes, at step 1510, selecting, for abitstream representation of a current block of visual media data, anintra prediction mode from at least one of adjacent or non-adjacentspatial neighboring blocks to the current block.

In some embodiments, and in the context of Example 6, the at least oneof the adjacent spatial neighboring blocks is an above neighboring blockrelative to a top-left position of the current block, or an aboveneighboring block relative to a center position of a first row of thecurrent block, or an above neighboring block relative to a top-rightposition of the current block, or a left neighboring block relative to atop-left position of the current block, or a left neighboring blockrelative to a center position of a first column of the current block, ora left neighboring block relative to a bottom-left position of thecurrent block.

The method 1500 includes, at step 1520, processing, based on the intraprediction mode, the bitstream representation to generate the currentblock.

FIG. 16 shows a flowchart of yet another example method forcross-component prediction. The method 1600 includes, at step 1610,generating, for a bitstream representation of a current block of visualmedia data, a first prediction block and a second prediction block thatare based on reconstructed samples from an image segment that comprisesthe current block.

The method 1600 includes, at step 1620, generating at least a portion ofa final prediction block based on a linear function of the first andsecond prediction blocks. In some embodiments, the final predictionblock is an average of the first and second prediction blocks.

In some embodiments, and in the context of Example 8, one portion of thefinal prediction block is based on the linear function of the first andsecond prediction blocks, and the remaining portion is copied directlyfrom the first or second prediction blocks. In some embodiments, theweights of the linear function are signaled in a sequence parameter set(SPS), a picture parameter set (PPS), a video parameter set (VPS), aslice header, a tile header, a group of coding tree units (CTUs), acoding unit (CU), a prediction unit (PU) or a transform unit (TU).

The method 1600 includes, at step 1630, processing, based on the finalprediction block, the bitstream representation to generate the currentblock.

5 Example Implementations of the Disclosed Technology

FIG. 17 is a block diagram of a video processing apparatus 1700. Theapparatus 1700 may be used to implement one or more of the methodsdescribed herein. The apparatus 1700 may be embodied in a smartphone,tablet, computer, Internet of Things (IoT) receiver, and so on. Theapparatus 1700 may include one or more processors 1702, one or morememories 1704 and video processing hardware 1706. The processor(s) 1702may be configured to implement one or more methods (including, but notlimited to, methods 1400, 1500 and 1600) described in the presentdocument. The memory (memories) 1704 may be used for storing data andcode used for implementing the methods and techniques described herein.The video processing hardware 1706 may be used to implement, in hardwarecircuitry, some techniques described in the present document.

In some embodiments, the video coding methods may be implemented usingan apparatus that is implemented on a hardware platform as describedwith respect to FIG. 17.

FIG. 18 is a block diagram showing an example video processing system1800 in which various techniques disclosed herein may be implemented.Various implementations may include some or all of the components of thesystem 1800. The system 1800 may include input 1802 for receiving videocontent. The video content may be received in a raw or uncompressedformat, e.g., 8 or 10 bit multi-component pixel values, or may be in acompressed or encoded format. The input 1802 may represent a networkinterface, a peripheral bus interface, or a storage interface. Examplesof network interface include wired interfaces such as Ethernet, passiveoptical network (PON), etc. and wireless interfaces such as Wi-Fi orcellular interfaces.

The system 1800 may include a coding component 1804 that may implementthe various coding or encoding methods described in the presentdocument. The coding component 1804 may reduce the average bitrate ofvideo from the input 1802 to the output of the coding component 1804 toproduce a coded representation of the video. The coding techniques aretherefore sometimes called video compression or video transcodingtechniques. The output of the coding component 1804 may be eitherstored, or transmitted via a communication connected, as represented bythe component 1806. The stored or communicated bitstream (or coded)representation of the video received at the input 1802 may be used bythe component 1808 for generating pixel values or displayable video thatis sent to a display interface 1810. The process of generatinguser-viewable video from the bitstream representation is sometimescalled video decompression. Furthermore, while certain video processingoperations are referred to as “coding” operations or tools, it will beappreciated that the coding tools or operations are used at an encoderand corresponding decoding tools or operations that reverse the resultsof the coding will be performed by a decoder.

Examples of a peripheral bus interface or a display interface mayinclude universal serial bus (USB) or high definition multimediainterface (HDMI) or Displayport, and so on. Examples of storageinterfaces include SATA (serial advanced technology attachment), PCI,IDE interface, and the like. The techniques described in the presentdocument may be embodied in various electronic devices such as mobilephones, laptops, smartphones or other devices that are capable ofperforming digital data processing and/or video display.

FIG. 19 shows a flowchart of an example method 1900 for video coding inaccordance with the disclosed technology. The method 1900 includes, atoperation 1910, selecting, for a conversion between a current block ofvisual media data and a bitstream representation of the current block, afirst intra prediction mode based on at least a first set of historyintra coding information that includes statistical information of a setof intra prediction modes. The method 1900 also includes, at operation1920, performing the conversion based on the first intra predictionmode.

In some embodiments, the conversion includes encoding the current blockto generate the bitstream representation. In some embodiments, theconversion includes decoding the bitstream representation to generatethe current block. In some embodiments, the statistical intra codinginformation comprises a number of occurrences of each of the set ofintra prediction modes over a time duration. In some embodiments, thestatistical intra coding information comprises a number of occurrencesof a part of the set of intra prediction modes over a time duration.

In some embodiments, the method further includes, comprising, afterperforming the conversion, updating the set of history intra codinginformation. In some embodiments, the updating comprises accumulatingthe number of occurrences of the first intra prediction mode by k in thefirst set of history intra coding information, k being a positiveinteger. In some embodiments, a subsequent block of the current block isprocessed based on the updated history intra coding information.

In some embodiments, each intra prediction mode in the first set ofhistory intra coding information is associated with a limit for thenumber of occurrences. In some embodiments, the method includes, afterthe number of occurrences reaches the limit, reducing the number ofoccurrences based on a predefined rule. In some embodiments, thepredefined rule comprises dividing the number of occurrences by a firstpredefined value, or subtracting a second predefined value from thenumber of occurrences.

In some embodiments, the method includes constructing a most probablemode (MPM) list based on at least the set of intra prediction modes. Insome embodiments, the method includes adding a subset of the set ofintra prediction modes to the MPM list. In some embodiments, the methodincludes adding intra prediction modes associated with spatial neighborsof the current block to the MPM list, adding a subset of the set ofintra prediction modes to the MPM list, and pruning the MPM list. Insome embodiments, the method includes adding derived intra predictionmodes to the MPM list. In some embodiments, the method includesincreasing a size of the MPM list. In some embodiments, the methodincludes reordering the MPM list based on the set of history intracoding information. In some embodiments, the MPM list is reordered basedon a descending order of the statistical information of the set of intraprediction modes.

In some embodiments, the current block comprises a luma component and achroma component. The first set of history intra coding information isused for intra-mode coding the luma component. In some embodiments, asecond set of history intra coding information is used for intra-modecoding the chroma component. In some embodiments, the luma component isbased on a first coding tree and the chroma component is based on asecond coding tree different from the first coding tree.

In some embodiments, selecting the first intra prediction mode isfurther based on intra prediction modes of spatial neighbors of thecurrent block or intra prediction modes of non-adjacent neighbors of thecurrent block. In some embodiments, the history intra coding informationis stored in a table.

FIG. 20 shows a flowchart of another example method 2000 for videoprocessing in accordance with the disclosed technology. The method 2000includes, at operation 2010, selecting, for a conversion between acurrent block of visual media data and a bitstream representation of thecurrent block, a first intra prediction mode based on at least intraprediction modes associated with non-adjacent neighbors of the currentblock. The method 2000 also includes, at operation 2020, performing theconversion based on the first intra prediction mode.

In some embodiments, the conversion includes encoding the current blockto generate the bitstream representation. In some embodiments, theconversion includes decoding the bitstream representation to generatethe current block.

In some embodiments, the method further includes constructing, for theconversion between the current block and the bitstream representation ofthe current block, a most probable mode (MPM) list based on the intraprediction modes associated with the non-adjacent neighbors of thecurrent block. The conversion is performed further based on the MPMlist. In some embodiments, constructing the MPM list includes addingintra prediction modes associated with spatial neighbors of the currentblock to the MPM list and pruning the MPM list. In some embodiments,constructing the MPM list includes adding derived intra prediction modesto the MPM list. In some embodiments, the method includes increasing asize of the MPM list.

In some embodiments, the method includes reordering intra predictionmodes that are not in the MPM list according to the intra predictionmodes associated with the non-adjacent neighbors of the current block.In some embodiments, the intra prediction modes associated with thenon-adjacent neighbors of the current block are associated with a higherpriority for the reordering.

FIG. 21 shows a flowchart of an example method 2100 for video processingin accordance with the disclosed technology. The method 2100 includes,at operation 2110, selecting, for a conversion between a current blockof visual media data and a bitstream representation of the currentblock, an intra prediction mode based on at least one of spatialneighboring blocks of the current block. The at least one of the spatialneighboring blocks is different from a first block that is located to aleft of a first row of the current block and a second block that islocated above a first column of the current block. The method 2100 alsoincludes, at operation 2110, performing the conversion based on theintra prediction mode.

In some embodiments, the conversion comprises encoding the current blockto generate the bitstream representation. In some embodiments, theconversion comprises decoding the bitstream representation to generatethe current block.

In some embodiments, the at least one of the spatial neighboring blocksincludes a block adjacent to the current block. In some embodiments, theat least one of the spatial neighboring blocks includes a blocknon-adjacent to the current block. In some embodiments, the at least oneof the adjacent spatial neighboring blocks includes a block adjacent toa top-left position of the current block. In some embodiments, the atleast one of the adjacent spatial neighboring blocks includes a blockadjacent to a center position of a first row of the current block. Insome embodiments, the at least one of the adjacent spatial neighboringblocks includes a block adjacent to a center position of a first columnof the current block.

In some embodiments, the at least one of the spatial neighboring blocksis selected based on one or more dimensions of the current block. Insome embodiments, the current block has a square shape, and the at leastone of the adjacent spatial neighboring blocks includes a block adjacentto a top-left position of the current block. The selected intraprediction mode can be added to a most-probable-mode list of the currentblock.

In some embodiments, the current block has a non-square shape, and theat least one of the spatial neighboring blocks includes a block adjacentto a center position of a first row of the current block or a blockadjacent to a center position of a first column of the current block.

FIG. 22 shows a flowchart of an example method 2200 for video processingin accordance with the disclosed technology. The method 2200 includes,at operation 2210, generating a final prediction block for a conversionbetween a current block of visual media data and a bitstreamrepresentation of the current block. The method 2200 includes, atoperation 2220, performing the conversion based on the final predictionblock. At least a portion of the final prediction block is generatedbased on a combination of a first prediction block and a secondprediction block that are based on reconstructed samples from an imagesegment that comprises the current block. In some embodiments, theconversion comprises encoding the current block to generate thebitstream representation. In some embodiments, the conversion comprisesdecoding the bitstream representation to generate the current block.

In some embodiments, the first prediction block is generated accordingto an intra prediction mode. In some embodiments, the second predictionblock is generated based on a motion vector pointing to the imagesegment. In some embodiments, the second prediction block is generatedbased on an intra block copy technology in which the reconstructedsamples pointed by the motion vector are copied. In some embodiments,the second prediction block is generated based on an intra block copytechnology in which motion compensation is applied to the reconstructedsamples pointed by the motion vector before applying an in-loopfiltering operation.

In some embodiments, the reconstructed samples are generated before anin-loop filtering operation is applied. In some embodiments, a remainingportion of the final prediction block is copied directly from the firstor second prediction block. In some embodiments, the combination of thefirst and the second prediction blocks comprises a linear function ofthe first and second prediction blocks. In some embodiments, the linearfunction comprises an average of the first and second prediction blocks.In some embodiments, the linear function includes a weighted linearfunction of the first and the second prediction blocks. In someembodiments, different weights are applied for different positionsrelative to the first or the second prediction block. In someembodiments, different weights are applied for different intraprediction modes. In some embodiments, weights of the weighted linearfunction are signaled in a sequence parameter set (SPS), a pictureparameter set (PPS), a video parameter set (VPS), a slice header, a tileheader, a group of coding tree units (CTUs), a coding unit (CU), aprediction unit (PU) or a transform unit (TU). In some embodiments,weights of the weighted linear function are predefined.

In some embodiments, the image segment includes a picture, a slice or atile. In some embodiments, the first prediction block and the secondprediction block are generated for a selected color component of thecurrent block. In some embodiments, the selected color componentincludes a luma color component. In some embodiments, the firstprediction block and the second prediction block are generated upondetermining that a size or a shape of the current block satisfies apredefined criterion.

FIG. 23 shows a flowchart of an example method 2300 for video processingin accordance with the disclosed technology. The method 2300 includes,at operation 2310, maintaining a first table of motion candidates duringa conversion between a current block of visual media data and abitstream representation of the current block. The method 2300 includes,at operation 2320, determining, based on at least the first table ofmotion candidates, motion information for the current block that iscoded using an intra block copy (IBC) mode in which at least one motionvector is directed at an image segment that comprises the current block.The method 2300 also includes, at operation 2330, performing theconversion based on the motion information. In some embodiments, theconversion includes encoding the current block to generate the bitstreamrepresentation. In some embodiments, the conversion includes decodingthe bitstream representation to generate the current block.

In some embodiments, the method includes selectively updating, based ona rule, the first table of motion candidates using the motioninformation for the conversion. In some embodiments, the rule specifiesadding the motion information to the first table of motion candidates.In some embodiments, the rule specifies excluding the motion informationfrom the first table of motion candidates.

In some embodiments, the method includes updating, based on the motioninformation for the current block, the first table of motion candidatesafter the conversion. In some embodiments, the method includesperforming a second conversion between a second block of visual mediadata and the bitstream representation of the second block using theupdated first table of motion candidates. The second block can be codedusing the intra block copy mode. In some embodiments, the first table ofmotion candidates includes only motion information from blocks that arecoded using the intra block copy mode. In some embodiments, the firsttable of motion candidates is only used for processing blocks that arecoded using the intra block copy mode. In some embodiments, the firsttable of motion candidates includes motion information from blocks thatare coded using the intra block copy mode and excludes motioninformation from blocks that are coded using other techniques. In someembodiments, the first table of motion candidates includes motioninformation from blocks that are coded using the intra block copy modeand motion information from blocks that are coded using othertechniques. In some embodiments, the second conversion is performedusing motion candidates associated with IBC coded blocks in the updatedfirst table of motion candidates.

In some embodiments, the method includes maintaining, for a thirdconversion between a third block of visual media data and the bitstreamrepresentation of the third block, a second table of motion candidates,performing the third conversion between the third block and thebitstream representation of the third block based on the second table ofmotion candidates. In some embodiments, the third conversion isperformed without using the updated first table of motion candidates. Insome embodiments, the second table includes motion information fromblocks that are encoded using other techniques that are different thanthe IBC mode. In some embodiments, the third block is coded using atechnique that is different than the IBC mode. In some embodiments, thetechnique includes an inter mode. In some embodiments, the methodincludes updating the second table of motion candidates using motioninformation for the third conversion. In some embodiments, the methodincludes performing a fourth conversion between an IBC coded block basedon the updated first table and performing a fifth conversion between aninter coded block based on the updated second table.

In some embodiments, the method includes comparing a motion candidateassociated with the current block with a number of entries in the firsttable. The first table is updated based on the comparing. In someembodiments, the number of entries corresponds to all entries in thefirst tables. In some embodiments, the number of entries is m, m beingan integer, and the m entries are last m entries in the first table. Insome embodiments, updating the first table comprises adding the motioncandidate to the first table. In some embodiments, the first tableincludes no duplicated entries.

In some embodiments, the method includes determining that the currentblock is further coded using an IBC merge mode or an IBC Advanced MotionVector Prediction (AMVP) mode. In some embodiments, the method includesdetermining a list of motion candidates for the current block bycombining the first table of motion candidates and a set of adjacent ornon-adjacent candidates of the current block and performing theconversion based on the list. In some embodiments, the combiningcomprises checking m candidates from the set of adjacent or non-adjacentcandidates, and checking n motion candidates from the first table ofmotion candidates, wherein m and n are positive integers. In someembodiments, the method includes adding at least one of the m candidatesfrom the set of adjacent or non-adjacent candidates to the list ofmotion candidates upon determining that the list is not full. In someembodiments, the method includes adding at least one of the n motioncandidates from the first table of motion candidates to the list ofmotion candidates upon determining that the list is not full.

In some embodiments, the combining comprises checking candidates fromthe set of adjacent or non-adjacent candidates and the first table ofmotion candidates in an interleaved manner. In some embodiments, themethod includes checking a candidate from the set of adjacent ornon-adjacent candidates prior to checking a motion candidate from thefirst table. In some embodiments, the checking comprises checking acandidate from the set of adjacent or non-adjacent candidates andreplacing, based on a coding characteristic of an adjacent ornon-adjacent block associated with the adjacent or non-adjacentcandidate, the candidate with a motion candidate from the first table ofmotion candidates. In some embodiments, the coding characteristic of theadjacent or non-adjacent block indicates that the adjacent ornon-adjacent block is located outside a predefined range. In someembodiments, the coding characteristic of the adjacent or non-adjacentblock indicates that the adjacent or non-adjacent block is intra coded.In some embodiments, adding a motion candidate further comprising addingthe motion candidate to the list upon determining that the motioncandidate from the first table of motion candidates is different from atleast one candidate from the set of adjacent or non-adjacent candidates.In some embodiments, the set of adjacent or non-adjacent candidate has ahigher priority than the first table of motion candidates.

In some embodiments, the list of motion candidates comprises multiplemerge candidates used for IBC merge-coded blocks. In some embodiments,the list of motion candidates comprises multiple AMVP candidates usedfor IBC AMVP-coded blocks. In some embodiments, the list of motioncandidates comprises intra prediction modes used for intra-coded blocks.In some embodiments, a size of the first table is pre-defined orsignaled in the bitstream representation. In some embodiments, the firsttable is associated with a counter that indicates a number of availablemotion candidates in the first table.

In some embodiments, the method includes resetting the first tablebefore processing a new slice, a new LCU row, or a new tile. In someembodiments, the counter is set to zero before processing a new slice, anew LCU row, or a new tile.

From the foregoing, it will be appreciated that specific embodiments ofthe presently disclosed technology have been described herein forpurposes of illustration, but that various modifications may be madewithout deviating from the scope of the invention. Accordingly, thepresently disclosed technology is not limited except as by the appendedclaims.

Implementations of the subject matter and the functional operationsdescribed in this patent document can be implemented in various systems,digital electronic circuitry, or in computer software, firmware, orhardware, including the structures disclosed in this specification andtheir structural equivalents, or in combinations of one or more of them.Implementations of the subject matter described in this specificationcan be implemented as one or more computer program products, i.e., oneor more modules of computer program instructions encoded on a tangibleand non-transitory computer readable medium for execution by, or tocontrol the operation of, data processing apparatus. The computerreadable medium can be a machine-readable storage device, amachine-readable storage substrate, a memory device, a composition ofmatter effecting a machine-readable propagated signal, or a combinationof one or more of them. The term “data processing unit” or “dataprocessing apparatus” encompasses all apparatus, devices, and machinesfor processing data, including by way of example a programmableprocessor, a computer, or multiple processors or computers. Theapparatus can include, in addition to hardware, code that creates anexecution environment for the computer program in question, e.g., codethat constitutes processor firmware, a protocol stack, a databasemanagement system, an operating system, or a combination of one or moreof them.

A computer program (also known as a program, software, softwareapplication, script, or code) can be written in any form of programminglanguage, including compiled or interpreted languages, and it can bedeployed in any form, including as a stand-alone program or as a module,component, subroutine, or other unit suitable for use in a computingenvironment. A computer program does not necessarily correspond to afile in a file system. A program can be stored in a portion of a filethat holds other programs or data (e.g., one or more scripts stored in amarkup language document), in a single file dedicated to the program inquestion, or in multiple coordinated files (e.g., files that store oneor more modules, sub programs, or portions of code). A computer programcan be deployed to be executed on one computer or on multiple computersthat are located at one site or distributed across multiple sites andinterconnected by a communication network.

The processes and logic flows described in this specification can beperformed by one or more programmable processors executing one or morecomputer programs to perform functions by operating on input data andgenerating output. The processes and logic flows can also be performedby, and apparatus can also be implemented as, special purpose logiccircuitry, e.g., an FPGA (field programmable gate array) or an ASIC(application specific integrated circuit).

Processors suitable for the execution of a computer program include, byway of example, both general and special purpose microprocessors, andany one or more processors of any kind of digital computer. Generally, aprocessor will receive instructions and data from a read only memory ora random access memory or both. The essential elements of a computer area processor for performing instructions and one or more memory devicesfor storing instructions and data. Generally, a computer will alsoinclude, or be operatively coupled to receive data from or transfer datato, or both, one or more mass storage devices for storing data, e.g.,magnetic, magneto optical disks, or optical disks. However, a computerneed not have such devices. Computer readable media suitable for storingcomputer program instructions and data include all forms of nonvolatilememory, media and memory devices, including by way of examplesemiconductor memory devices, e.g., EPROM, EEPROM, and flash memorydevices. The processor and the memory can be supplemented by, orincorporated in, special purpose logic circuitry.

It is intended that the specification, together with the drawings, beconsidered example only. As used herein, the use of “or” is intended toinclude “and/or”, unless the context clearly indicates otherwise.

While this patent document contains many specifics, these should not beconstrued as limitations on the scope of any invention or of what may beclaimed, but rather as descriptions of features that may be specific toparticular embodiments of particular inventions. Certain features thatare described in this patent document in the context of separateembodiments can also be implemented in combination in a singleembodiment. Conversely, various features that are described in thecontext of a single embodiment can also be implemented in multipleembodiments separately or in any suitable subcombination. Moreover,although features may be described above as acting in certaincombinations and even initially claimed as such, one or more featuresfrom a claimed combination can in some cases be excised from thecombination, and the claimed combination may be directed to asubcombination or variation of a subcombination.

Similarly, while operations are depicted in the drawings in a particularorder, this should not be understood as requiring that such operationsbe performed in the particular order shown or in sequential order, orthat all illustrated operations be performed, to achieve desirableresults. Moreover, the separation of various system components in theembodiments described in this patent document should not be understoodas requiring such separation in all embodiments.

Only a few implementations and examples are described and otherimplementations, enhancements and variations can be made based on whatis described and illustrated in this patent document.

What is claimed is:
 1. A method for video processing, comprising:generating a final prediction block for a conversion between a currentblock of a video and a bitstream of the video; and performing theconversion based on the final prediction block, wherein at least aportion of the final prediction block is generated based on acombination of a first prediction block and a second prediction blockthat are based on reconstructed samples from an image segment thatcomprises the current block, wherein the second prediction block isgenerated based on an intra block copy technology in which motioncompensation is applied to the reconstructed samples pointed by a motionvector before applying an in-loop filtering operation.
 2. The method ofclaim 1, wherein the conversion includes encoding the current block intothe bitstream.
 3. The method of claim 1, wherein the conversion includesdecoding the current block from the bitstream.
 4. The method of claim 1,wherein the first prediction block is generated according to an intraprediction mode.
 5. The method of claim 1, wherein the motion vectorpoints to the image segment.
 6. The method of claim 5, wherein thereconstructed samples pointed by the motion vector are copied.
 7. Themethod of claim 1, wherein: the reconstructed samples are generatedbefore the in-loop filtering operation is applied.
 8. The method ofclaim 1, wherein a remaining portion of the final prediction block iscopied directly from the first or second prediction block.
 9. The methodof claim 1, wherein the combination of the first and the secondprediction blocks comprises a linear function of the first and secondprediction blocks, wherein the linear function comprises an average ofthe first and second prediction blocks or a weighted linear function ofthe first and the second prediction blocks.
 10. The method of claim 9,wherein different weights are applied for different positions relativeto the first or the second prediction block.
 11. The method of claim 10,wherein different weights are applied for different intra predictionmodes.
 12. The method of claim 1, wherein the image segment includes apicture, a slice or a tile.
 13. The method of claim 1, wherein the firstprediction block and the second prediction block are generated for aselected color component of the current block.
 14. The method of claim13, wherein the selected color component includes a luma colorcomponent.
 15. The method of claim 1, wherein the first prediction blockand the second prediction block are generated upon determining that asize or a shape of the current block satisfies a predefined criterion.16. An apparatus for video processing, comprising a processor and anon-transitory memory with instructions thereon, wherein theinstructions upon execution by the processor, cause the processor to:generate a final prediction block for a conversion between a currentblock of a video and a bitstream of the video; and perform theconversion based on the final prediction block, wherein at least aportion of the final prediction block is generated based on acombination of a first prediction block and a second prediction blockthat are based on reconstructed samples from an image segment thatcomprises the current block, wherein the second prediction block isgenerated based on an intra block copy technology in which motioncompensation is applied to the reconstructed samples pointed by a motionvector before applying an in-loop filtering operation.
 17. The apparatusof claim 16, wherein the reconstructed samples pointed by the motionvector are copied.
 18. The apparatus of claim 16, wherein thecombination of the first and the second prediction blocks comprises alinear function of the first and second prediction blocks, wherein thelinear function comprises a weighted linear function of the first andthe second prediction blocks.
 19. The apparatus of claim 18, whereindifferent weights are applied for different positions relative to thefirst or the second prediction block.
 20. A non-transitorycomputer-readable recording medium storing a bitstream of a video whichis generated by a method performed by a video processing apparatus,wherein the method comprises: generating a final prediction block for aconversion between a current block of the video and the bitstream of thevideo; and generating the bitstream based on the final prediction block,wherein at least a portion of the final prediction block is generatedbased on a combination of a first prediction block and a secondprediction block that are based on reconstructed samples from an imagesegment that comprises the current block, wherein the second predictionblock is generated based on an intra block copy technology in whichmotion compensation is applied to the reconstructed samples pointed by amotion vector before applying an in-loop filtering operation.